Weighted Geometric Grammars for Object Detection in Context

This thesis addresses the problem of detecting objects in images of complex scenes. Strong patterns exist in the types and spatial arrangements of objects that occur in scenes, and we seek to exploit these patterns to improve detection performance. We introduce a novel formalism—weighted geometric grammars (WGGs)—for flexibly representing and recognizing combinations of objects and their spatial relationships in scenes. We adapt the structured perceptron algorithm to parameter learning in WGG models, and develop a set of original clustering-based algorithms for structure learning. We then demonstrate empirically that WGG models, with parameters and structure learned automatically from data, can outperform a standard object detector. This thesis also contributes three new fully-labeled datasets, in two domains, to the scene understanding community. Thesis Supervisor: Leslie Pack Kaelbling Title: Professor Thesis Supervisor: Tomás Lozano-Pérez Title: Professor